The present invention relates to a method and a filtering system for filtering edge directions in an edge orientation map detected from an interlaced or non-interlaced scan. The filtered edge directions can then be used in an image interpolation process to preserve the smoothness of the edges in the interpolated image.
Referring to FIG. 1, the meaning of the term xe2x80x9cedge directionxe2x80x9d can be seen. Along the edge direction, pixels"" luminance values remain constant or change gradually, however, across the edge direction, pixels"" luminance values change sharply.
Image interpolation is usually performed in both image de-interlacing and image scaling applications. There are various kinds of methods used for image interpolation. With most methods, one common problem is the degradation of edges such as the effects of serrate lines associated with the interpolated image. To overcome this problem, it is desirable to find edge directions in the image and to do image interpolation along those directions so that the smoothness of the edges can be preserved.
There have been a large number of methods published on edge detection, including edge location detection and edge direction detection. However, most edge detection methods are sensitive to image noise. In addition, the neighborhood of a given pixel in an image can have numerous cases for an edge detection method to process. As a result, it is common, if not inevitable, for an edge detection system to provide a certain percentage of false edge locations or wrong edge directions.
Meanwhile, the accuracy of the edge directions serves a crucial role when the directions are used in image applications such as de-interlacing or scaling. In edge direction based image de-interlacing, the edge direction may be detected at the position of each pixel to be interpolated. Then for edge pixels, interpolation can be performed along the edge directions to preserve the smoothness of the edges in the interpolated image. The same idea can be applied to image scaling as well. However, in these applications if a detected edge is false or if the edge direction is not correct, obvious error may be introduced into the interpolated image. Therefore, when edge directions are utilized in these kinds of applications, false edges or edges with wrong directions must be excluded before the edge information is used.
It is accordingly an object of the invention to provide a method and a filtering system for filtering edge directions, which overcome the above-mentioned disadvantages of the prior art methods and apparatus of this general type.
In particular, it is an object of the invention to provide a filtering method that can be applied to an edge orientation map obtained from an edge direction detection system. The filtering method should keep accurate edge directions and filter out false edges or edges with wrong directions.
To accomplish the object of the invention, a direction length filter is provided for performing a direction length filtering process. The direction length filter is used to filter the edge direction along the direction indicated by the edge direction. Along the current direction, neighboring edge directions are checked to see whether their orientations are similar to the current direction being checked. If the neighboring directions have very different orientations from the current direction, the current direction is filtered out. The assumption for this filtering is that a valid edge direction should be able to be extended and should therefore have a certain minimum length.
A direction width filter for performing a direction width filtering process may also be provided. The width of a direction in an edge orientation map is defined as the number of consecutive neighboring positions along the horizontal direction where the edge directions are close to the current one. The direction width filter is used to filter out edge directions in the edge orientation map that are too thin. If along the horizontal direction there are a certain minimum number of consecutive neighboring edge directions that have similar orientations to the current one, then the current direction is kept. Otherwise, the current direction is filtered out.
A direction smoother for performing a direction smoothing process may also be provided. The direction smoother is basically a low pass filter. When the direction smoother is applied to directions in an edge orientation map, the filter can smooth the changing of neighboring edge directions. This process is found to be effective in improving the visual quality of an image that is interpolated based on edge directions.
With the foregoing and other objects in view there is provided, in accordance with the invention, a method for filtering an edge direction. The method includes steps of: in a given row in an edge orientation map, selecting a given pixel having an edge direction; finding neighboring upper pixels having neighboring upper edge directions for the given pixel by traversing along the edge direction of the given pixel from the given pixel to a neighboring upper row of the edge orientation map; finding neighboring lower pixels having neighboring lower edge directions for the given pixel by traversing along the edge direction of the given pixel from the given pixel to a neighboring lower row of the edge orientation map; determining whether the edge direction of the given pixel can be extended to the neighboring upper row by checking an orientation similarity between the edge direction of the given pixel and the neighboring upper edge directions; determining whether the edge direction of the given pixel can be extended to the neighboring lower row by checking an orientation similarity between the edge direction of the given pixel and the neighboring lower edge directions; if the edge direction of the given pixel can be extended to both the neighboring upper row and the neighboring lower row, then assuming that the edge direction of the given pixel is valid; if the edge direction of the given pixel cannot be extended to the neighboring lower row, but the edge direction of the given pixel can be extended to one of the neighboring upper pixels, and a neighboring upper edge direction at the one of the neighboring upper pixels can be extended to a further neighboring upper row of the edge orientation map, then assuming that the edge direction of the given pixel is valid; if the edge direction of the given pixel cannot be extended to the neighboring upper row, but the edge direction of the given pixel can be extended to one of the neighboring lower pixels, and a neighboring lower edge direction at the one of the neighboring lower pixels can be extended to a further neighboring lower row of the edge orientation map, then assuming that the edge direction of the given pixel is valid; and otherwise, assuming that the edge direction of the given pixel is not valid.
In accordance with an added feature of the invention, the edge direction of the given pixel (x,y) is represented by d(x,y), and the step of finding the neighboring upper pixels includes using equations:
ƒ(x,y)=d(xxe2x88x922,cu(x,y)),
ƒl(x,y)=d(xxe2x88x922,cu(x,y)xe2x88x921), and
ƒr(x,y)=d(xxe2x88x922,cu(x,y)+1), where cu(x,y)=y+d(x,y)*2.
In addition, the step of finding the neighboring lower pixels includes using equations:
g(x,y)=d(x+2,cd(x,y)),
gl(x,y)=d(x+2,cd(x,y)xe2x88x921), and
xe2x80x83gr(x,y)=d(x+2,cd(x,y)+1), where cd(x,y)=yxe2x88x92d(x,y)*2.
In accordance with an additional feature of the invention, the step of checking the orientation similarity between the edge direction of the given pixel and the neighboring upper edge directions includes:
a) defining a checking function chkup(x,y) as follows:       chkup    ⁡          (              x        ,        y            )        =      {                            1                          if                                                    "LeftBracketingBar"                                                d                  ⁡                                      (                                          x                      ,                      y                                        )                                                  -                                  f                  ⁡                                      (                                          x                      ,                      y                                        )                                                              "RightBracketingBar"                        ≤                          T              1                                                or                                                  xe2x80x83                                                xe2x80x83                                                              "LeftBracketingBar"                                                d                  ⁡                                      (                                          x                      ,                      y                                        )                                                  -                                                      f                    l                                    ⁡                                      (                                          x                      ,                      y                                        )                                                              "RightBracketingBar"                        ≤                          T              1                                                or                                                  xe2x80x83                                                xe2x80x83                                                              "LeftBracketingBar"                                                d                  ⁡                                      (                                          x                      ,                      y                                        )                                                  -                                                      f                    r                                    ⁡                                      (                                          x                      ,                      y                                        )                                                              "RightBracketingBar"                        ≤                          T              1                                                            xe2x80x83                                                0                                      xe2x80x83                                    otherwise                                      xe2x80x83                              
b) concluding that the edge direction of the given pixel can be extended to the neighboring upper row if chkup(x,y) equals one; and
the step of checking the orientation similarity between the edge direction of the given pixel and the neighboring lower edge directions includes:
a) defining a checking function chkdn(x,y) as follows:       chkdn    ⁡          (              x        ,        y            )        =      {                            1                          if                                                    "LeftBracketingBar"                                                d                  ⁡                                      (                                          x                      ,                      y                                        )                                                  -                                  g                  ⁡                                      (                                          x                      ,                      y                                        )                                                              "RightBracketingBar"                        ≤                          T              1                                                or                                                  xe2x80x83                                                xe2x80x83                                                              "LeftBracketingBar"                                                d                  ⁡                                      (                                          x                      ,                      y                                        )                                                  -                                                      g                    l                                    ⁡                                      (                                          x                      ,                      y                                        )                                                              "RightBracketingBar"                        ≤                          T              1                                                or                                                  xe2x80x83                                                xe2x80x83                                                              "LeftBracketingBar"                                                d                  ⁡                                      (                                          x                      ,                      y                                        )                                                  -                                                      g                    r                                    ⁡                                      (                                          x                      ,                      y                                        )                                                              "RightBracketingBar"                        ≤                          T              1                                                            xe2x80x83                                                0                                      xe2x80x83                                    otherwise                                      xe2x80x83                              
b) concluding that the edge direction of the given pixel can be extended to the neighboring lower row if chkdn(x,y) equals one.
Here, T1 is a threshold value chosen to be greater than zero and small enough to verify correlations of neighboring edge directions, but not small enough such that slightly different directions will remain.
In accordance with another feature of the invention, the method includes:
defining a checking function chkup2(x,y) as:       chkup2    ⁡          (              x        ,        y            )        =      {                            1                          if                                      (                                                            "LeftBracketingBar"                                                            d                      ⁡                                              (                                                  x                          ,                          y                                                )                                                              -                                          f                      ⁡                                              (                                                  x                          ,                          y                                                )                                                                              "RightBracketingBar"                                ≤                                                      T                    1                                    ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                                      chkup                    ⁡                                          (                                                                        x                          -                          2                                                ,                                                                              c                            u                                                    ⁡                                                      (                                                          x                              ,                              y                                                        )                                                                                              )                                                                                  =              1                        )                                    or                                                  xe2x80x83                                                xe2x80x83                                                (                                                            "LeftBracketingBar"                                                            d                      ⁡                                              (                                                  x                          ,                          y                                                )                                                              -                                                                  f                        l                                            ⁡                                              (                                                  x                          ,                          y                                                )                                                                              "RightBracketingBar"                                ≤                                                      T                    1                                    ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                  chkup                  ⁢                                      (                                                                  x                        -                        2                                            ,                                                                                                    c                            u                                                    ⁡                                                      (                                                          x                              ,                              y                                                        )                                                                          -                        1                                                              )                                                              =              1                        )                                    or                                                  xe2x80x83                                                xe2x80x83                                                (                                                            "LeftBracketingBar"                                                            d                      ⁡                                              (                                                  x                          ,                          y                                                )                                                              -                                                                  f                        r                                            ⁡                                              (                                                  x                          ,                          y                                                )                                                                              "RightBracketingBar"                                ≤                                                      T                    1                                    ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                  chkup                  ⁢                                      (                                                                  x                        -                        2                                            ,                                                                                                    c                            u                                                    ⁡                                                      (                                                          x                              ,                              y                                                        )                                                                          +                        1                                                              )                                                              =              1                        )                                                xe2x80x83                                                0                                      xe2x80x83                                    otherwise                                      xe2x80x83                              
defining a checking function chkdn2(x,y) as:       chkdn2    ⁡          (              x        ,        y            )        =      {                            1                          if                                      (                                                            "LeftBracketingBar"                                                            d                      ⁡                                              (                                                  x                          ,                          y                                                )                                                              -                                          g                      ⁡                                              (                                                  x                          ,                          y                                                )                                                                              "RightBracketingBar"                                ≤                                                      T                    1                                    ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                                      chkdn                    ⁡                                          (                                                                        x                          +                          2                                                ,                                                                              c                            d                                                    ⁡                                                      (                                                          x                              ,                              y                                                        )                                                                                              )                                                                                  =              1                        )                                    or                                                  xe2x80x83                                                xe2x80x83                                                (                                                            "LeftBracketingBar"                                                            d                      ⁡                                              (                                                  x                          ,                          y                                                )                                                              -                                                                  g                        l                                            ⁡                                              (                                                  x                          ,                          y                                                )                                                                              "RightBracketingBar"                                ≤                                                      T                    1                                    ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                                      chkdn                    ⁡                                          (                                                                        x                          +                          2                                                ,                                                                                                            c                              d                                                        ⁡                                                          (                                                              x                                ,                                y                                                            )                                                                                -                          1                                                                    )                                                                                  =              1                        )                                    or                                                  xe2x80x83                                                xe2x80x83                                                (                                                            "LeftBracketingBar"                                                            d                      ⁡                                              (                                                  x                          ,                          y                                                )                                                              -                                                                  g                        r                                            ⁡                                              (                                                  x                          ,                          y                                                )                                                                              "RightBracketingBar"                                ≤                                                      T                    1                                    ⁢                                      xe2x80x83                                    ⁢                  and                  ⁢                                      xe2x80x83                                    ⁢                                      chkdn                    ⁡                                          (                                                                        x                          +                          2                                                ,                                                                                                            c                              d                                                        ⁡                                                          (                                                              x                                ,                                y                                                            )                                                                                +                          1                                                                    )                                                                                  =              1                        )                                                xe2x80x83                                                0                                      xe2x80x83                                    otherwise                                      xe2x80x83                              
if the checking function chkup2(x,y) takes a value of one, concluding that the neighboring upper edge direction at the one of the neighboring upper pixels can be extended to the further neighboring upper row of the edge orientation map; and if the checking function chkdn2(x,y) takes a value of one, concluding that the neighboring lower edge direction at the one of the neighboring lower pixels can be extended to the further neighboring lower row of the edge orientation map.
In accordance with a further feature of the invention, if the edge direction of the given pixel is within a threshold value from two horizontally neighboring directions in the given row, then a final conclusion is made that the edge direction of the given pixel is valid.
In accordance with a further added feature of the invention, the method includes: checking a width of the edge direction of the given pixel by determining an orientation similarity between the edge direction of the given pixel and two horizontally neighboring directions in the given row.
In accordance with a further additional feature of the invention, the edge direction of the given pixel (x,y) is represented by d(x,y); and the step of checking the width of the edge direction of the given pixel includes using equations:
max(|d(x,y)xe2x88x92d(x,yxe2x88x921)|,|d(x,y+1)xe2x88x92d(x,y)|)xe2x89xa6T2,
max(|d(x,y)xe2x88x92d(x,yxe2x88x921)|,|d(x,yxe2x88x921)xe2x88x92d(x,yxe2x88x922)|)xe2x89xa6T2, and
max(|d(x,y+1)xe2x88x92d(x,y)|,|d(x,y+2)xe2x88x92d(x,y+1)|xe2x89xa6T2.
Here, T2 is a threshold value chosen to be greater than zero and small enough to verify correlations of neighboring edge directions, but not small enough such that slightly different directions will remain.
In accordance with yet an added feature of the invention, the method includes: performing a filtering process to smooth a transition between neighboring edge directions.
In accordance with yet an additional feature of the invention, the method includes: using a three-tap filter for the filtering process; and implementing the three-tap filter using equation: ds(x,y)=(d(x,yxe2x88x921)+2*d(x,y)+d(x,y+1))/4. Here, ds(x,y) is a filtered output direction.
With the foregoing and other objects in view there is also provided, in accordance with the invention, an edge direction filtering system including: a direction length filter for extending a current edge direction to neighboring upper rows and to neighboring lower rows in an edge orientation map. The direction length filter is for checking an orientation similarity between the current edge direction and neighboring edge directions located along the current edge direction. The direction length filter indicates that the current edge direction is valid if the current edge direction can be extended to a certain number of neighboring rows in the edge orientation map.
In accordance with an added feature of the invention, a direction width filter is provided for filtering the current edge direction along a horizontal direction. The direction width filter is for checking an orientation similarity between the current edge direction and neighboring edge directions on a row with the current edge direction in an edge orientation map. The direction width filter indicates that the current edge direction is valid if a certain number of consecutive ones of the neighboring edge directions on the row with the current edge direction have a similar orientation to the current edge direction.
In accordance with a concomitant feature of the invention, a low pass filter is provided for smoothing transitions between horizontally neighboring edge directions of the edge orientation map.